System and method for controlled sharing of consumable resources in a computer cluster

ABSTRACT

In one embodiment, a method includes empirically analyzing, by a computer cluster comprising a plurality of computers, a set of active reservations and a current set of consumable resources belonging to a class of consumable resources. Each active reservation is of a managed task type and comprises a group of one or more tasks task requiring access to a consumable resource of the class. The method further includes, based on the empirically analyzing, clocking the set of active reservations each clocking cycle. The method also includes, responsive to the clocking, sorting, by the computer cluster, a priority queue of the set of active reservations.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application claims priority from, and incorporates byreference the entire disclosure of, U.S. Provisional Patent ApplicationNo. 61/717,798 filed on Oct. 24, 2012. This patent applicationincorporates by reference the entire disclosure of a U.S. patentapplication Ser. No. 13/690,526, filed on Nov. 30, 2012.

BACKGROUND

1. Technical Field

The present invention relates generally to scheduling and moreparticularly, but not by way of limitation, to systems and methods forcontrolled sharing of consumable resources in a computer cluster.

2. History of Related Art

A multi-tenant computer cluster such as, for example, a Software as aService (SaaS) cloud, often uses computing resources (e.g., hardware andsoftware resources) to perform services for customers. The computingresources can include, for example, computers (e.g., physical computersor virtual machines (VMs)), raw (block) and file-based storage,firewalls, load balancers, internet protocol (IP) addresses, virtuallocal area networks (VLANs), software bundles, and computing platformsthat typically include an operating system, a programming-languageexecution environment, a database, and a web server. Services generallyinvolve use of at least a portion of the computing resources for thebenefit of the customer. The services can include, for example,emergency notification, accounting, collaboration, customer relationshipmanagement (CRM), management information systems (MIS), enterpriseresource planning (ERP), invoicing, human resource management (HRM),content management (CM), service-desk management, and/or many otherservices.

Oftentimes, the multi-tenant computer cluster receives customer requestsfor service randomly responsive to needs that arise unpredictably. Inaddition, the customer requests for service frequently spawn otherrequests for service. Therefore, all requests are traditionally managedin a queue and serviced in a first-in first-out (FIFO) manner. As thequeue becomes backlogged, the FIFO manner of servicing results in anunequal distribution of the computing resources across customers.Because the requests are serviced sequentially, customers with morerequests are awarded a greater share of the computing resources thancustomers with fewer requests. The distribution of the computingresources across all customers is not generally controllable.

Moreover, as the value and use of information continues to increase,individuals and businesses seek additional ways to process and storeinformation. One option available to users is information handlingsystems. An information handling system generally processes, compiles,stores, and/or communicates information or data for business, personal,or other purposes thereby allowing users to take advantage of the valueof the information. Because technology and information handling needsand requirements vary between different users or applications,information handling systems may also vary regarding what information ishandled, how the information is handled, how much information isprocessed, stored, or communicated, and how quickly and efficiently theinformation may be processed, stored, or communicated. The variations ininformation handling systems allow for information handling systems tobe general or configured for a specific user or specific use such asfinancial transaction processing, airline reservations, enterprise datastorage, or global communications. In addition, information handlingsystems may include a variety of hardware and software components thatmay be configured to process, store, and communicate information and mayinclude one or more computer systems, data storage systems, andnetworking systems.

SUMMARY OF THE INVENTION

In one embodiment, a method includes, via at least one process on acomputer cluster comprising a plurality of computers, identifying acurrent set of consumable resources that fulfill a resource need of amanaged task type. The method further includes deriving, by the at leastone process, a set of active reservations of the managed task type. Inaddition, the method includes apportioning, by the at least one process,the current set of consumable resources among the set of activereservations to yield a balanced-utilization partitioning scheme. Thebalanced-utilization partitioning scheme includes a flow-controlclocking weight for each active reservation.

In one embodiment, an information handling system includes a computercluster comprising a plurality of computers. The computer cluster has atleast one flow-control instance resident and executing thereon. For eachof the at least one flow-control instance, the computer cluster isoperable to identify a current set of consumable resources that fulfilla resource need of a managed task type. The computer cluster is furtheroperable to derive a set of active reservations of the managed tasktype. In addition, the computer cluster is operable to apportion thecurrent set of consumable resources among the set of active reservationsto yield a balanced-utilization partitioning scheme. Thebalanced-utilization partitioning scheme comprising a flow-controlclocking weight for each active reservation.

In one embodiment, a computer-program product includes a computer-usablemedium having computer-readable program code embodied therein. Thecomputer-readable program code adapted to be executed to implement amethod. The method includes identifying a current set of consumableresources that fulfill a resource need of a managed task type. Themethod further includes deriving a set of active reservations of themanaged task type. In addition, the method includes apportioning thecurrent set of consumable resources among the set of active reservationsto yield a balanced-utilization partitioning scheme. Thebalanced-utilization partitioning scheme includes a flow-controlclocking weight for each active reservation.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the method and apparatus of the presentinvention may be obtained by reference to the following DetailedDescription when taken in conjunction with the accompanying Drawingswherein:

FIG. 1 illustrates a computer cluster;

FIG. 2 illustrates a flow-control scheme;

FIG. 3 illustrates a flow-control instance;

FIG. 4 illustrates a process that may be executed by a flow-controlinstance;

FIG. 5 illustrates a derivation of reservations from tasks;

FIG. 6 illustrates an exemplary balanced-utilization partitioningscheme;

FIG. 7 illustrates an implementation of a balanced-utilizationpartitioning scheme;

FIG. 8 illustrates an exemplary balanced-utilization partitioningscheme;

FIG. 9 illustrates an implementation of a balanced-utilizationpartitioning scheme;

FIG. 10 illustrates a balanced-utilization partitioning scheme;

FIG. 11 illustrates an implementation of a balanced-utilizationpartitioning scheme;

FIG. 12 illustrates a process for calculating an inner stationarydistribution;

FIG. 13 illustrates clocking functionality of a flow-control instance;

FIG. 14 illustrates a process for decrementing a wait-time;

FIG. 15 illustrates a collection of interactive processes that may beexecuted by a flow-control instance; and

FIG. 16 illustrates a collection of interactive processes that may beexecuted by a flow-control instance.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS OF THE INVENTION

In various embodiments, customers can be served more equitably andcontrollably using systems and methods described herein. In a typicalembodiment, flow control is instituted on a computer cluster by defininga class of consumable resources and establishing a framework governingutilization of the consumable resources by customers. For purposes ofthis patent application, a computer cluster is a set ofloosely-connected computers, virtual or physical, that work togethersuch that the computers can be viewed as a single system. In variousembodiments, a computer cluster can provide a cloud environment such as,for example, a SaaS cloud environment.

A consumable resource, as used herein, refers to a limited resource thatis accessible to a computer cluster. For example, consumable resourcescan include, but are not limited to, available memory, centralprocessing units (CPUs), free space on a file system, network bandwidth,floating software licenses, voice-port hardware (e.g., text-to-speechvoice ports) and access to a computing process. For simplicity,consumable resources may be referenced periodically herein as resources.In particular, with respect to access to a computing process, thecomputing process may be, for example, a bottleneck point in an overallbusiness process. It should be appreciated that consumable resources canbe either internal or external to a given computer cluster. It shouldfurther be appreciated that consumable resources can also be used toabstract human workflow. For example, in some embodiments, principlesdescribed herein are implemented in a call center that receives incomingcalls to be serviced by call-center employees. In these embodiments,either the call-center employees or voice-communication channels staffedby such employees can be considered consumable resources that arecontrolled and managed as described herein.

A service, as used herein, is a semantic process or a combination ofsemantic processes that can be performed for the benefit of a customer.Services are generally requested by customers. A semantic process, asused herein, refers to one or more tasks performed by a computer. Tasksof a given task type, which can span multiple semantic processes, mayutilize one or more consumable resources from a specific class ofconsumable resources. A class of consumable resources, as used herein,refers to an aggregation of like consumable resources that can fulfill aresource need, for example, of the given task type. In such cases, aflow-control instance can be utilized to control sharing of the specificclass of consumable resources relative to tasks of the given task type.For example, a class of consumable resource could include a batch oftasks or task groups that can be executed at a given time or allowed toaccess a computing process. A flow-control instance, as used herein, issoftware that is configured to manage and control sharing of aparticular class of consumable resources among a plurality of customers.

FIG. 1 illustrates a computer cluster 100 that is operable to provideone or more services to customers. The computer cluster 100 includes acomputer 102(1), a computer 102(2), a computer 102(3), and a database106. For convenient reference, the computer 102(1), the computer 102(2),and the computer 102(3) may be referenced collectively as computers 102.In various embodiments, the computers 102 can be virtual computers,physical computers, or a combination thereof. For illustrative purposes,the computers 102 are shown to include the computer 102(1), the computer102(2), and the computer 102(3). However, one of ordinary skill in theart will appreciate that, in practice, the computers 102 can include anynumber of physical and/or virtual computers. As shown, the computers 102and the database 106 are operable to communicate over a network 108.

In a typical embodiment, each of the computers 102 is operable toexecute one or more semantic processes related to the provision of theone or more services by the computer cluster 100. In particular, thecomputer 102(1) executes a process 104 a(1), the computer 102(2)executes a process 104 b(1) and a process 104 c(1), and the computer102(3) executes a process 104 a(2), a process 104 b(2), and a process104 c(2). In a typical embodiment, the process 104 a(1) and the process104 a(2) are identical processes that have been replicated on each ofthe computer 102(1) and the computer 102(3), respectively. Likewise, theprocess 104(b)(1) and the process 104(b)(2) are identical processes thathave been replicated on each of the computer 102(2) and the computer102(3), respectively. Similarly, the process 104(c)(1) and the process104(c)(2) are identical processes that have been replicated on each ofthe computer 102(2) and the computer 102(3), respectively.

Operationally, the one or more services provided by the computer cluster100 can be initiated in a variety of ways. For example, in a typicalembodiment, services can be requested by one of the customers via, forexample, an interface provided over a public network such as, forexample, the Internet. Once a service is initiated, the initiatedservice may encompass semantic processes selected from the processes102(a)(1), 102(a)(2), 102(b)(1), 102(b)(2), 102(c)(1), and 102(c)(2).The semantic processes of the initiated service generally include tasksto be executed by the computers 102. In various embodiments, eachservice, semantic process, and task can spawn other services, semanticprocesses, and tasks, respectively, so that the initiated serviceresults, for example, in many thousands of tasks. In a typicalembodiment, some of those tasks may be of a task type that requiresaccess to a specific class of consumable resources. In variousembodiments, the computer cluster 100 controls sharing of the specificclass of consumable resources via a flow-control instance that isexecuted on one or more of the computers 102. Examples of flow-controlinstances that may be utilized will be described with respect to theensuing Figures.

FIG. 2 illustrates a flow-control scheme 200 involving process threads216, 218, and 220. The flow-control scheme 200 illustrates thatflow-control instances generally serve a gating function. The processthreads 216, 218, and 220 each represent business logic that may beexecuted in parallel by a computer cluster such as, for example, thecomputer cluster 100 of FIG. 1, in the provision of one or more servicesto a customer. The process threads 216, 218, and 220 can each includeone or more semantic processes, which semantic processes can eachfurther include one or more tasks.

The process thread 216 includes a logical path 216(1), a flow-controlinstance 216(2), and a semantic process 216(3). As shown, flow of theprocess thread 216 is gated by the flow-control instance 216(2). Theflow-control instance 216(2) controls access to a class of consumableresources needed by a task of the process thread 216. Therefore, forwardprogress in the process thread 216, and execution of the semanticprocess 216(3), is conditioned upon the flow-control instance 216(2)providing consumable-resource access to the task and the task executing.After the task has executed, the semantic process 216(3) can beinitiated.

The process thread 218 includes a logical path 218(1), a flow-controlinstance 218(2), a flow-control instance 218(3), a flow-control instance218(4), and a semantic process 218(5). As shown, flow of the processthread 218 is gated by the flow-control instances 218(2), 218(3), and218(4). The flow-control instances 218(2), 218(3), and 218(4) eachcontrol access to a distinct class of consumable resources needed bythree distinct tasks of the process thread 218. For purposes ofillustration, the flow-control instances 218(2), 218(3), and 218(4) maybe considered to represent a first task, a second task, and a thirdtask, respectively. Initially, forward progress in the process thread218 is conditioned upon: (1) the first task being granted resourceaccess by the flow-control instance 218(2) and executing; and (2) thesecond task being granted resource access by the flow-control instance218(3) and executing. Therefore, as shown, the third task cannot executeuntil the first task and the second task are granted consumable-resourceaccess by the flow-control instances 218(2) and 218(3), respectively,and execute.

Once the first task and the second task have executed, forward progressin the process thread 218, and execution of the semantic process 218(5),is conditioned upon the flow-control instance 218(4) providingconsumable-resource access to the third task and the third taskexecuting. After the third task has executed, the semantic process218(5) can be initiated. In that way, the process thread 218 utilizesthree flow-control instances in the provision of the one or moreservices.

The process thread 220 includes a logical path 220(1), a flow-controlinstance 220(2), and a semantic process 220(3). As shown, flow of theprocess thread 220 is gated by the flow-control instance 220(2). Theflow-control instance 220(2) controls access to a class of consumableresources needed by a task of the process thread 220. Therefore, forwardprogress in the process thread 220, and execution of the semanticprocess 220(3), is conditioned upon the flow-control instance 220(2)providing consumable-resource access to the task and the task executing.After the task has executed, the semantic process 220(3) can beinitiated.

FIG. 3 illustrates a flow-control instance 300 that may be implementedon a computer cluster such as, for example, the computer cluster 100 ofFIG. 1. The flow-control instance 300 performs controlled sharing 316 ofa current set of consumable resources 310 for a given task type beingmanaged by the flow-control instance 300 (hereinafter, “managed tasktype”). The current set of consumable resources 310 corresponds to aclass of consumable resources required by the managed task type. Theflow-control instance 300 enforces the controlled sharing 316 among acurrent set of active customers 312 according to a balanced-utilizationpartitioning scheme 314. In a typical embodiment, the flow-controlinstance 300 is resident and executing on at least one computer of thecomputer cluster.

In a typical embodiment, the flow-control instance 300 manages tasks ofthe managed task type by grouping tasks into reservations. For example,each reservation can include those tasks that have a same task key. Thetask key usually defines a set of characteristics that justify groupingtasks into a same reservation. The set of characteristics can include,for example, a service ID for a requested service with which the task isassociated, a customer ID corresponding to a requesting customer, apriority level assigned to the task, and the like. Reservations will bedescribed in greater detail with respect to the ensuing Figures.

In a typical embodiment, the current set of consumable resources 310includes a defined quantity of consumable resources for which theflow-control instance 300 is responsible for enforcing the controlledsharing 316. The defined quantity can dynamically change duringexecution of the flow-control instance 300. For example, consumableresources may become available or unavailable to the flow-controlinstance 300 and, correspondingly, be included in or excluded from thecurrent set of consumable resources 310. For illustrative purposes, thecurrent set of consumable resources 310 is shown to include consumableresources 310(1), 310(2), and 310(3), which resources may be consideredto represent an exemplary snapshot-in-time of the current set ofconsumable resources 310.

The current set of active customers 312 includes customers that, at agiven point in time, have at least one reservation of the managed tasktype that has not been processed to completion by the computer cluster(i.e., customers having active reservations). In a typical embodiment,the current set of active customers 312 changes dynamically duringexecution of the flow-control instance 300. As new requests for serviceare received and corresponding reservations are created, customers whodo not already have at least one active reservation of the managed tasktype are added to the current set of active customers 312. Asreservations are processed to completion, customers having no activereservations of the managed task type are removed from the current setof active customers 312.

The balanced-utilization partitioning scheme 314 establishes how thecurrent set of consumable resources 310 should be distributed to andconsumed by the current set of active customers 312. More particularly,the balanced-utilization partitioning scheme 314 defines logicalpartitions relative to a totality of the current set of consumableresources 310. Each logical partition of the balanced-utilizationpartitioning scheme 314 is assigned a resource percentage representing aproportion of a given set of consumable resources (e.g., the current setof consumable resources 310 or a subset thereof) that, ideally, shouldbe allocated to the logical partition. For example, in variousembodiments, a total number of partitions may equal a number of activecustomers in the current set of active customers 312. In various otherembodiments, customers may be allowed to prioritize their tasks. Inthese embodiments, the total number of partitions could vary between thetotal number of active customers and the total number of activecustomers multiplied by a total number of priority levels enabled by theflow-control instance.

For illustrative purposes, the balanced-utilization partitioning scheme314 is shown to include partitioning schemes 314(1), 314(2), and 314(3),which schemes may be considered an exemplary snapshot-in-time of thebalanced-utilization partitioning scheme 314. Like the current set ofconsumable resources 310 and the current set of active customers 312,the balanced-utilization partitioning scheme 314 dynamically changesduring execution of the flow-control instance 300 based on, for example,the current set of active customers 312 and a current set of activereservations of the managed task type. As the balanced-utilizationpartitioning scheme 314 changes, new resource percentages are computedand assigned.

FIG. 4 illustrates a process 400 that may be executed, for example, by aflow-control instance that is resident and executing on at least onecomputer of a computer cluster. Various portions of the process 400 maybe executed as part of the controlled sharing 316 of FIG. 3. It shouldbe appreciated that steps 402-410 of the process 400 are shown toexecute sequentially for purposes of illustrating how tasks logicallyprogress from being grouped into reservations to being assigned freeresources. Although the process 400 can execute sequentially as shown,in a typical embodiment, steps 402-410 each represent subprocesses thatexecute in parallel on the computer cluster. In addition, as describedbelow, various steps of the process 400 execute continuously to controlsharing of a current set of consumable resources among a current set ofactive reservations. The process 400 begins at step 402.

At step 402, the flow-control instance derives reservations. Derivationof reservations involves grouping tasks that have, for example, a sametask key, into reservations. As indicated, step 402 executes repeatedlydue to constant creation of new tasks of a managed task type. Derivationof reservations will be described in greater detail with respect to FIG.5. After each execution of step 402, the process 400 proceeds to step404. At step 404, the flow-control instance empirically analyzes thecurrent set of active reservations and the current set of consumableresources. In a typical embodiment, step 404 yields abalanced-utilization partitioning scheme such as, for example, thebalanced-utilization partitioning scheme 314 of FIG. 3. Examples ofempirical analysis that can occur at step 402 will be described ingreater detail with respect to FIGS. 6-12. After execution of step 404,the process 400 proceeds to step 406.

At step 406, the flow-control instance performs clocking of the currentset of active reservations. In a typical embodiment, the flow-controlinstance maintains a virtual clock that, every clocking cycle, initiatesa virtual pendulum for each reservation of the current set of activereservations. The clocking cycle can be, for example, a configurableunit of wall time (e.g., seconds or milliseconds) or a configurablenumber of CPU cycles. The initiation of the virtual pendulums results ineach reservation being “clocked” according to a flow-control clockingweight that is specific to the reservation. Clocking of reservationswill be described in more detail with respect to FIGS. 13-14. Asindicated, step 406 executes repeatedly for each clocking cycle. Afterstep 406, the process 400 proceeds to step 408.

At step 408, the flow-control instance determines whether a consumableresource of the current set of consumable resources is free (i.e.,available for use). Whenever a consumable resource is free, the process400 proceeds to step 410. As indicated in FIG. 4, once initiated, step408 executes continuously as long as reservations exist in the priorityqueue and there are free resources available to assign. At step 410, theflow-control instance assigns the free consumable resource to amost-needy resource in the priority queue. Examples of activities thatcan occur at steps 408 and 410 will be described in greater detail withrespect to FIGS. 15-16. After step 410, the process 400 ends.

FIG. 5 illustrates a derivation 500 that can be used to derivereservations from tasks. The derivation 500 illustrates functionalitythat, in a typical embodiment, can be performed as part of step 402 ofFIG. 4. One or more flow-control instances 518 are shown to derivereservations from a task table 502. For purposes of illustration, theone or more flow-control instances 518 are shown to include aflow-control instance 518 a, a flow-control instance 518 b, aflow-control instance 518 c, and a flow-control instance 518 d. In atypical embodiment, the task table 502 is a comprehensive listing of alltasks in a computer cluster such as, for example, the computer cluster100 of FIG. The task table 502 is typically maintained separately fromany given flow-control instance. The task table 502 is regularly updatedto add new tasks that are created and to remove tasks that have beencompleted.

The one or more flow-control instances 518 manage tasks of a managedtask type as described with respect to FIG. 2. Tasks having a same taskkey, as determined by information from the task table, are grouped intoa common reservation. As described above, the task key usually defines aset of characteristics that justify grouping tasks into a samereservation. The set of characteristics can include, for example, aservice ID for a requested service with which the task is associated, acustomer ID corresponding to a requesting customer, a priority levelassigned to the task, and the like.

In a typical embodiment, flow-control performance can be enhanced when,as illustrated, the one or more flow-control instances 518 include morethan one flow-control instance to manage tasks of the managed task type.In these embodiments, flow control is distributed across the one or moreflow-control instances 518, with each flow-control instance operating toderive reservations and perform flow control in a similar fashion. Itshould be appreciated that the one or more flow-control instances 518need not perform steps of a process such as, for example, the process400 of FIG. 4, at identical intervals. Rather, each flow-controlinstance can execute such a process independently and thus derivereservations at different intervals. The task table 502 operates as acommon source of tasks for the one or more flow-control instances 518.Each of the flow-control instances 518 can be viewed as a cooperativeforce that results in resources being assigned to reservations derivedfrom the task table 502. Therefore, when flow control for the managedtask type is distributed as illustrated in FIG. 5, this cooperativeeffect causes the one or more flow-control instances 518 to exhibit theemergent property.

FIGS. 6-12 illustrate empirical analysis of reservations in greaterdetail. In a typical embodiment, functionality described with respect toFIGS. 6-12 can be performed as part of step 404 of the process 400 ofFIG. 4. Empirical analysis typically includes generation of abalanced-utilization partitioning scheme such as, for example, thebalanced-utilization partitioning scheme 314 of FIG. 3. In a typicalembodiment, the balanced-utilization partitioning scheme utilizesprinciples of stationary distributions. As used herein, a stationarydistribution refers to a set of values that sum to one. As describedbelow, the values can be, for example, percentages of a set ofconsumable resources. For purposes of this description, the stationarydistribution of one represents one-hundred percent of a given set orsubset of consumable resources. In that way, as described below, abalance of resource consumption can be precisely maintained.

FIG. 6 illustrates an exemplary balanced-utilization partitioning scheme600 that may be utilized by a flow-control instance. For example, thebalanced-utilization partitioning scheme 600 may be implemented as thebalanced-utilization partitioning scheme 314 of FIG. 3. Thebalanced-utilization partitioning scheme 600 includes a partition 602, apartition 604, a partition 606, and a partition 608 that each correspondto an active customer (i.e., four active customers). According to thebalanced-utilization scheme 600, all active customers at a given a pointin time share a class of consumable resources equally.

FIG. 7 illustrates an implementation 700 of the balanced-utilizationpartitioning scheme 600 of FIG. 6. The implementation 700 includes acurrent set of consumable resources 702 that is apportioned amongcustomer-specific reservation sets 706(1), 706(2), 706(3), 706(4), and706(5) (collectively, customer-specific reservation sets 706). As shown,the current set of consumable resources 702 is apportioned to thecustomer-specific reservation sets 706 according to a stationarydistribution 704. The current set of consumable resources 702 includes Nresources.

Each of the customer-specific reservation sets 706 is an aggregation ofactive reservations for a particular customer (for each of fivecustomers as shown). In particular, the customer-specific reservationset 706(1) includes reservations 711 a, 711 b, and 711 c. Thecustomer-specific reservation set 706(2) includes reservations 711 d and711 e. The customer-specific reservation set 706(3) includesreservations 711 f, 711 g, 711 h, and 711 i. The customer-specificreservation set 706(4) includes reservation 711 j. Finally, thecustomer-specific reservation set 706(5) includes reservation 711 k. Forconvenient reference, the reservations 711 a-k may be referencedcollectively as reservations 711.

The stationary distribution 704 functions to distributecustomer-specific resource percentages 708(1), 708(2), 708(3), 708(4),and 708(5) to the customer-specific reservation sets 706(1), 706(2),706(3), 706(4), and 706(5), respectively. For convenient reference, thecustomer-specific resource percentages 708(1)-708(5) may be referencedcollectively as customer-specific resource percentages 708. Incompliance with the stationary distribution 704, the customer-specificresource percentages 708 are values that, when summed, equal one. Forexample, when applying the balanced-utilization scheme 600 of FIG. 6,the current set of consumable resources 702 is apportioned equally.According to this example, each of the customer-specific reservationsets 708 is apportioned one-fifth of the current set of consumableresources 702. As such, each of the customer-specific resourcepercentages 708 would equal 0.2 in decimal form.

After the stationary distribution 704 is applied, stationarydistributions 710(1), 710(2), 710(3), 710(4), and 710(5) are applied(collectively, stationary distributions 710). The stationarydistributions 710 apportion the customer-specific resource percentages708 to individual reservations of the customer-specific reservation sets706. More particularly, the stationary distributions 710(1), 710(2),710(3), 710(4), and 710(5) serve to distribute reservation-specificresource percentages 712 a-712 c, 712 d-712 e, 712 f-712 i, 712 j, and712 k, respectively. In a typical embodiment, the stationarydistributions 710 effect an equal apportionment of the customer-specificresource percentages 708 across each reservation set of thecustomer-specific reservation sets 706.

More particularly, the stationary distribution 710(1) apportions thecustomer-specific resource percentage 708(1) to the reservations 711a-711 c (i.e., the customer-specific reservation set 706(1)). In thisfashion, the reservation-specific resource percentages 712 a, 712 b, and712 c are distributed to the reservations 711 a, 711 b, and 711 c,respectively. The reservation-specific resource percentages 712 a-712 cthereby represent proportions of the customer-specific resourcepercentage 708(1) that, according to the stationary distribution 710(1),collectively sum to one.

The stationary distribution 710(2) apportions the customer-specificresource percentage 708(2) to the reservations 711 d-711 e (i.e., thecustomer-specific reservation set 706(2)). In this fashion, thereservation-specific resource percentages 712 d and 712 e aredistributed to the reservations 711 d and 711 e, respectively. Thereservation-specific resource percentages 712 d and 712 e therebyrepresent proportions of the customer-specific resource percentage708(2) that, according to the stationary distribution 710(2),collectively sum to one.

The stationary distribution 710(3) apportions the customer-specificresource percentage 708(3) to the reservations 711 f-711 i (i.e., thecustomer-specific reservation set 706(3)). In this fashion, thereservation-specific resource percentages 712 f, 712 g, 712 h, and 712 iare distributed to the reservations 711 f, 711 g, 711 h, and 711 i,respectively. The reservation-specific resource percentages 712 f-712 ithereby represent proportions of the customer-specific resourcepercentage 708(3) that, according to the stationary distribution 710(3),collectively sum to one.

The stationary distribution 710(4) apportions the customer-specificresource percentage 708(4) to the reservation 711 j (i.e., thecustomer-specific reservation set 706(4)). In this fashion, thereservation-specific resource percentage 712 j is distributed to thereservation 711 j. The reservation-specific resource percentage 712 jthereby represents a proportion of the customer-specific resourcepercentage 708(4). Since the customer-specific reservation set 706(4)includes only the reservation 711 j, according to the stationarydistribution 710(4), the reservation-specific resource percentage 712 jwill generally equal one.

The stationary distribution 710(5) apportions the customer-specificresource percentage 708(5) to the reservation 711 k (i.e., thecustomer-specific reservation set 706(5)). In this fashion, thereservation-specific resource percentage 712 k is distributed to thereservation 711 k. The reservation-specific resource percentage 712 kthereby represents a proportion of the customer-specific resourcepercentage 708(5). Since the customer-specific reservation set 706(5)includes only the reservation 711 k, according to the stationarydistribution 710(5), the reservation-specific resource percentage 712 kwill generally equal one.

In a typical embodiment, the stationary distributions 710 effect anequal distribution of the customer-specific resource percentages 708across each reservation set of the customer-specific reservation sets706. For example, the customer-specific reservation set 706(1) includesthree reservations, i.e., the reservations 711 a, 711 b, and 711 c. Thereservation-specific resource percentages 712 a, 712 b, and 712 c shouldthus each equal one-third. The customer-specific reservation set 706(2)includes two reservations, i.e., the reservations 711 d and 711 e. Thereservation-specific resource percentages 712 d and 712 e should thuseach equal one-half. The customer-specific reservation set 706(3)includes four reservations, i.e., the reservations 711 f, 711 g, 711 h,and 711 i. The reservation-specific resource percentages 712 f, 712 g,712 h, and 712 i should thus each equal one-fourth. Thecustomer-specific reservation sets 706(4) and 706(5) each include asingle reservation, i.e., the reservations 711 j and 711 k,respectively. Therefore, as described above, the reservation-specificresource percentages 712 j and 712 k should each equal one.

After the stationary distributions 710 are applied, effectivedistributions 714 a-714 k are computed (collectively, effectivedistributions 714). As explained above, the reservation-specificresource percentages 712 are percentages of the customer-specificresource percentages 708 that should be allocated to the reservations711. The effective distributions 714 are, in effect, a translation ofthe reservation-specific resource percentages 712 into percentages ofthe current set of consumable resources 702. The effective distributions714 a-714 k are computed relative to the reservations 711 a-711 k,respectively.

Specifically, each of the effective distributions 714 can be computed asa product of a corresponding reservation-specific resource percentage(from the reservation-specific resource percentages 712) and acorresponding customer-specific resource percentage (from thecustomer-specific resource percentages 708). For example, the effectivedistribution 714 a can be computed as a product of thereservation-specific resource percentage 712 a and the customer-specificresource percentage 708(1). Table 1 below lists exemplary valuesrelative to the example of FIG. 7. It should be appreciated that theeffective distributions 714 should sum to one.

TABLE 1 EFFECTIVE DISTRIBUTION VALUE Effective distribution 714a0.0666666666666667 Effective distribution 714b 0.0666666666666667Effective distribution 714c 0.0666666666666667 Effective distribution714d 0.1 Effective distribution 714e 0.1 Effective distribution 714f0.05 Effective distribution 714g 0.05 Effective distribution 714h 0.05Effective distribution 714i 0.05 Effective distribution 714j 0.2Effective distribution 714k 0.2

Once the effective distributions 714 have been calculated, in a typicalembodiment, flow-control clocking weights 716 a-716 k are calculated(collectively, flow-control clocking weights 716). The flow-controlclocking weights 716 are, in effect, a translation of the effectivedistributions 714 into defined quantities of resources that should beallocated to each of the reservations 711. As explained in detail below,the flow-control clocking weights 716 can be calculated as products ofthe effective distributions 714 and a total number of resources in thecurrent set of consumable resources 702 (i.e., N).

In particular, the flow-control clocking weight 716 a equals theeffective distribution 714 a multiplied by N. The flow-control clockingweight 716 b equals the effective distribution 714 b multiplied by N.The flow-control clocking weight 716 c equals the effective distribution714 c multiplied by N. The flow-control clocking weight 716 d equals theeffective distribution 714 d multiplied by N. The flow-control clockingweight 716 e equals the effective distribution 714 e multiplied by N.The flow-control clocking weight 716 f equals the effective distribution714 f multiplied by N. The flow-control clocking weight 716 g equals theeffective distribution 714 g multiplied by N. The flow-control clockingweight 716 h equals the effective distribution 714 h multiplied by N.The flow-control clocking weight 716 i equals the effective distribution714 i multiplied by N. The flow-control clocking weight 716 j equals theeffective distribution 714 j multiplied by N. The flow-control clockingweight 716 k equals the effective distribution 714 k multiplied by N.

Each of the flow-control clocking weights 716 is representative of adefined number of resources from the current set of resources 702. Asillustrated, a sum 718 of the flow-control clocking weights 716 equalsthe total number of consumable resources (i.e., N). Therefore, each ofthe flow-control clocking weights may be expressed in fractional unitsof resources. As described in greater detail with respect to FIGS.13-14, the flow-clocking weights 716 enable the balanced-utilizationpartitioning scheme to be precisely executed on a per reservation basis.

FIG. 8 illustrates an exemplary balanced-utilization partitioning scheme800 that may be utilized by a flow-control instance. For example, thebalanced-utilization partitioning scheme 800 may be implemented as thebalanced-utilization partitioning scheme 314 of FIG. 3. Thebalanced-utilization partitioning scheme 800 includes two superpartitions: a high-priority super partition 802 and a regular-prioritysuper partition 804.

In a typical embodiment, the high-priority super partition 802 includesa subset of a current set of consumable resources that is allocated toeligible active customers. The eligible active customers may include,for example, customers who have elected to pay more for a higher qualityof service. In various embodiments, the subset of the current set ofconsumable resources may be expressed as a percentage, an integer, or inother ways that will be apparent to one of ordinary skill in the artafter reviewing the inventive principles described herein. Thehigh-priority super partition 802 is then shared among the activeeligible customers as described with respect to the balanced-utilizationpartitioning scheme 600 of FIG. 6.

In various embodiments, the regular-priority super partition 804includes all consumable resources of the current set of consumableresources except those consumable resources allocated to thehigh-priority super partition 802. In various embodiments, theregular-priority super partition 804 is shared among active customersnot eligible for the high-priority super partition 802 in the mannerdescribed with respect to FIG. 4. In various other embodiments, theregular-priority super partition 804 is shared among all activecustomers, including those eligible for the high-priority superpartition 802, in the manner described with respect to FIG. 4.

As described with respect to the balanced-utilization partitioningscheme 214 of FIG. 2A, the balanced-utilization partitioning scheme 800dynamically changes during execution of the flow-control instance.Correspondingly, in a typical embodiment, a size of the high-prioritysuper partition 802 and a size of the regular-priority super partition804 can also dynamically change. For example, if fewer than allconsumable resources allocated to the high-priority super partition 802are in use, those unused consumable resources can be allocated to theregular-priority super partition 804 until there are active eligiblecustomers for the high-priority super partition 802 requiring their use.

FIG. 9 illustrates an implementation 900 of the balanced-utilizationpartitioning scheme 800 of FIG. 8. The implementation 900 includes aregular-priority super partition 902 a and a high-priority superpartition 902 b. The regular-priority super partition 902 and thehigh-priority super partition 902 b generally correspond to theregular-priority super partition 804 and the high-priority superpartition 802, respectively, of FIG. 8. The regular-priority superpartition 902 a and the high-priority super partition 902 b eachcomprise a defined number of consumable resources (at a given time).

In a typical embodiment, the implementation 900 performs an equaldistribution 904 a of the regular-priority super partition 902 a acrosscustomers 901(1)-901(5). In a typical embodiment, the customers901(1)-901(5) represent those customers having active reservations(collectively, customers 901). Therefore, the customers 901(1)-901(5)are apportioned defined quantities 903(1)-903(5), respectively, ofresources from the regular-priority super partition 902 a. In a typicalembodiment, the defined quantities 903(1)-903(5), when summed, shouldequal a total number of resources contained within the regular-prioritysuper partition 902 a. According to the equal apportionment 904 a, thedefined quantities 903(1)-903(5) represent equal shares of theregular-priority super partition 902 a.

In a typical embodiment, the implementation 900 performs an unequaldistribution 904 b of the high-priority super partition 904 b acrossactive customers who are eligible for a higher quality of service, i.e.,the customers 901(2) and 901(4). In a typical embodiment, the customers901(2) and 902(4) are each eligible for predetermined resourcequantities 905(2) and 905(4), respectively, of the high-priority superpartition 904 b. According to the unequal apportionment 904 b, thepredetermined resource quantities 905(2) and 905(4) are not necessarilyequal, although equality is possible.

As a result of the equal apportionment 904 a and the unequalapportionment 904 b, the customers 901(1)-901(5) have resourceapportionments 906(1)-906(5), respectively. The resource apportionments906(1)-906(5) are typically sums of quantities apportioned via the equalapportionment 904 a and quantities apportioned via the unequalapportionment 904 b. In particular, as shown, the customers 901(1),901(3), and 901(5) are not eligible for the high-priority superpartition 902 b. Therefore, the resource apportionments 906(1), 906(3),and 906(5) equal the defined quantities 903(1), 903(3), and 903(5),respectively. Since the customer 901(2) is eligible for thehigh-priority super partition 902 b, the resource apportionment 906(2)equals a sum of the defined quantity 903(2) and the predeterminedresource quantity 905(2). In like manner, since the customer 901(4) isalso eligible for the high-priority super partition 902 b, the resourceapportionment 906(5) equals a sum of the defined quantity 903(4) and thepredetermined resource quantity 905(4).

Total apportioned resources 906 are an aggregation of the resourceapportionments 906(1)-906(5). As described with respect to FIG. 8, as anoptimization, resources of the high-priority super partition 902 b thatare not being utilized are allocated to the regular-priority superpartition 902 a. Therefore, in some embodiments, the total apportionedresources 906 will include a quantity of resources equal in number to acombined total number of resources in the regular-priority superpartition 902 a and the high-priority super partition 902 b. In variousother embodiments, it should appreciated that, by design, this may notbe the case. For purposes of this example, the quantity of resources inthe total apportioned resources 906 may be considered to be P.

Once the resource apportionments 906(1)-906(5) are computed, anumber-to-percentage distribution 908 can be applied. As describedabove, the resource apportionments 906(1)-906(5) represent quantities ofresources. The number-to-percentage distribution 908 operates to convertthe resource apportionments 906(1)-906(5) to percentages that aredistributed to the customer-specific reservation sets 910(1)-910(5)(collectively, customer-specific reservation sets 910), with eachreservation set constituting an aggregation of customer reservations.

In particular, the customer-specific reservation sets 910(1)-910(5) areaggregations of active reservations for the customers 901(1)-901(5),respectively. The customer-specific reservation set 910(1) includesreservations 915 a, 915 b, and 915 c. The customer-specific reservationset 910(2) includes reservations 915 d and 915 e. The customer-specificreservation set 910(3) includes reservations 915 f, 915 g, 915 h, and915 i. The customer-specific reservation set 910(4) includes reservation915 j. Finally, the customer-specific reservation set 910(5) includesreservation 915 k. For convenient reference, the reservations 915 a-kmay be referenced collectively as reservations 915.

The number-to-percentage distribution 908 functions to distributecustomer-specific resource percentages 912(1), 912(2), 912(3), 912(4),and 912(5) to the customer-specific reservation sets 910(1), 910(2),910(3), 910(4), and 910(5), respectively. For convenient reference, thecustomer-specific resource percentages 912(1)-912(5) may be referencedcollectively as customer-specific resource percentages 912. For example,the customer-specific resource percentages 912(1)-912(5) can equal theresource apportionments 906(1)-906(5), respectively, divided by a totalnumber of resources in the total apportioned resources 906. For example,if the resource apportionment 906(1) included five resources and thetotal apportioned resources 906 included one-hundred resources, thecustomer-specific resource percentage 912(1) would equal 0.05 in decimalform.

After the number-to-percentage distribution 908 is applied, stationarydistributions 914(1), 914(2), 914(3), 914(4), and 914(5) are applied(collectively, stationary distributions 914). The stationarydistributions 914 apportion the customer-specific resource percentages912 to individual reservations of the customer-specific reservation sets910. Specifically, the stationary distributions 914(1), 914(2), 914(3),914(4), and 914(5) serve to distribute reservation-specific resourcepercentages 916 a-916 c, 916 d-916 e, 916 f-916 i, 916 j, and 916 k,respectively. In a typical embodiment, the stationary distributions 914effect an equal apportionment of the customer-specific resourcepercentages 912 across each reservation set of the customer-specificreservation sets 910.

More particularly, the stationary distribution 914(1) apportions thecustomer-specific resource percentage 912(1) to the reservations 915a-915 c (i.e., the customer-specific reservation set 910(1)). In thisfashion, the reservation-specific resource percentages 916 a, 916 b, and916 c are distributed to the reservations 915 a, 915 b, and 915 c,respectively. The reservation-specific resource percentages 916 a-916 cthereby represent proportions of the customer-specific resourcepercentage 912(1) that, according to the stationary distribution 914(1),collectively sum to one.

The stationary distribution 914(2) apportions the customer-specificresource percentage 912(2) to the reservations 915 d-915 e (i.e., thecustomer-specific reservation set 910(2)). In this fashion, thereservation-specific resource percentages 916 d and 916 e aredistributed to the reservations 915 d and 915 e, respectively. Thereservation-specific resource percentages 916 d and 916 e therebyrepresent proportions of the customer-specific resource percentage912(2) that, according to the stationary distribution 914(2),collectively sum to one.

The stationary distribution 914(3) apportions the customer-specificresource percentage 912(3) to the reservations 915 f-915 i (i.e., thecustomer-specific reservation set 910(3)). In this fashion, thereservation-specific resource percentages 916 f, 916 g, 916 h, and 916 iare distributed to the reservations 915 f, 915 g, 915 h, and 915 i,respectively. The reservation-specific resource percentages 916 f-916 ithereby represent proportions of the customer-specific resourcepercentage 912(3) that, according to the stationary distribution 914(3),collectively sum to one.

The stationary distribution 914(4) apportions the customer-specificresource percentage 912(4) to the reservation 915 j (i.e., thecustomer-specific reservation set 910(4)). In this fashion, thereservation-specific resource percentage 916 j is distributed to thereservation 915 j. The reservation-specific resource percentage 916 jthereby represents a proportion of the customer-specific resourcepercentage 912(4). Since the customer-specific reservation set 910(4)includes only the reservation 915 j, according to the stationarydistribution 914(4), the reservation-specific resource percentage 916 jwill generally equal one.

The stationary distribution 914(5) apportions the customer-specificresource percentage 912(5) to the reservation 915 k (i.e., thecustomer-specific reservation set 910(5)). In this fashion, thereservation-specific resource percentage 916 k is distributed to thereservation 915 k. The reservation-specific resource percentage 916 kthereby represents a proportion of the customer-specific resourcepercentage 912(5). Since the customer-specific reservation set 910(5)includes only the reservation 915 k, according to the stationarydistribution 914(5), the reservation-specific resource percentage 916 kwill generally equal one.

In a typical embodiment, the stationary distributions 914 effect anequal distribution of the customer-specific resource percentages 912across each reservation set of the customer-specific reservation sets910. For example, the customer-specific reservation set 910(1) includesthree reservations, i.e., the reservations 915 a, 915 b, and 915 c. Thereservation-specific resource percentages 916 a, 916 b, and 916 c shouldthus each equal one-third. The customer-specific reservation set 910(2)includes two reservations, i.e., the reservations 915 d and 915 e. Thereservation-specific resource percentages 916 d and 916 e should thuseach equal one-half. The customer-specific reservation set 910(3)includes four reservations, i.e., the reservations 915 f, 915 g, 915 h,and 915 i. The reservation-specific resource percentages 916 f, 916 g,916 h, and 916 i should thus each equal one-fourth. Thecustomer-specific reservation sets 910(4) and 910(5) each include asingle reservation, i.e., the reservations 915 j and 915 k,respectively. Therefore, as described above, the reservation-specificresource percentages 916 j and 916 k should each equal one.

After the stationary distributions 914 are applied, effectivedistributions 918 a-918 k are computed (collectively, effectivedistributions 918). As explained above, the reservation-specificresource percentages 916 are percentages of the customer-specificresource percentages 912 that should be allocated to the reservations915. The effective distributions 918 are, in effect, a translation ofthe reservation-specific resource percentages 916 into percentages ofthe total apportioned resources 906. The effective distributions 918a-918 k are computed relative to the reservations 915 a-915 k,respectively.

Specifically, each of the effective distributions 918 can be computed asa product of a corresponding reservation-specific resource percentage(from the reservation-specific resource percentages 916) and acorresponding customer-specific resource percentage (from thecustomer-specific resource percentages 912). For example, the effectivedistribution 918 a can be computed as a product of thereservation-specific resource percentage 916 a and the customer-specificresource percentage 912(1).

Once the effective distributions 918 have been calculated, in a typicalembodiment, flow-control clocking weights 920 a-920 k are calculated(collectively, flow-control clocking weights 920). The flow-controlclocking weights 920 are, in effect, a translation of the effectivedistributions 918 into defined quantities of resources that should beallocated to each of the reservations 915. As explained in detail below,the flow-control clocking weights 920 can be calculated as products ofthe effective distributions 918 and a total number of apportionedresources in the total apportioned resources 906 (i.e., P).

In particular, the flow-control clocking weight 920 a equals theeffective distribution 918 a multiplied by P. The flow-control clockingweight 920 b equals the effective distribution 918 b multiplied by P.The flow-control clocking weight 920 c equals the effective distribution918 c multiplied by P. The flow-control clocking weight 920 d equals theeffective distribution 918 d multiplied by P. The flow-control clockingweight 920 e equals the effective distribution 918 e multiplied by P.The flow-control clocking weight 920 f equals the effective distribution918 f multiplied by P. The flow-control clocking weight 920 g equals theeffective distribution 918 g multiplied by P. The flow-control clockingweight 920 h equals the effective distribution 918 h multiplied by P.The flow-control clocking weight 920 i equals the effective distribution918 i multiplied by P. The flow-control clocking weight 920 j equals theeffective distribution 918 j multiplied by P. The flow-control clockingweight 920 k equals the effective distribution 918 k multiplied by P.

Each of the flow-control clocking weights 920 is representative of adefined number of resources from the total apportioned resources 906. Asillustrated, a sum 922 of the flow-control clocking weights 920 equalsthe total number of apportioned resources (i.e., P). Therefore, each ofthe flow-control clocking weights may be expressed in fractional unitsof resources. As described in greater detail with respect to FIGS.13-14, the flow-clocking weights 920 enable the balanced-utilizationpartitioning scheme to be precisely executed on a per reservation basis.

FIG. 10 illustrates a balanced-utilization partitioning scheme 1000 thatmay be utilized by a flow-control instance such as, for example, theflow-control instance 200 of FIG. 2A. For example, thebalanced-utilization partitioning scheme 1000 may be implemented as thebalanced-utilization partitioning scheme 214 of FIG. 2A. Thebalanced-utilization partitioning scheme 1000 includes a high-prioritysuper partition 1002 and a regular-priority super partition 1004. Thehigh-priority super partition 1002 operates as described with respect tothe high-priority super partition 802 of FIG. 8 and the high-prioritysuper partition 902 b of FIG. 9. The regular-priority super partition1004 operates as described with respect to the regular-priority superpartition 804 of FIG. 8 and the regular-priority super partition 902 aof FIG. 9. However, differently from FIGS. 8-9, the balanced-utilizationscheme 1000 additionally includes a low-priority super partition 1004 athat is a subset of the regular-priority super partition 1004.

The low-priority super partition 1004 a is a partition of dynamicallyvariable size and includes consumable resources of the regular-prioritysuper partition 1004 that are not in use at a given time. In a typicalembodiment, the low-priority super partition 1004 a allows a designatedclass of customers to pay a much lower price for service in exchange fora much lower quality of service. In addition, the low-priority superpartition 1004 a helps to maximize overall utilization of the currentset of consumable resources. Typically, the low-priority super partition1004 a is active only when the regular-priority super partition 1004 isnot being fully utilized by eligible active customers. Whenever thereare sufficient eligible active customers to fully utilize theregular-priority super partition 1004, the low-priority super partition1004 a is typically inactive.

In various embodiments, reservations can be associated with prioritylevels selected from a plurality of priority levels (e.g., prioritylevels one to nine). In a typical embodiment, the plurality of prioritylevels vary from low priority (e.g., level one) to high priority (e.g.,level nine). As described in more detail below, a priority levelassociated with a given reservation can be used to configurably vary acomputation of effective distributions.

FIG. 11 illustrates an implementation 1100 of a balanced-utilizationpartitioning scheme that utilizes priorities. The implementation 1100begins by apportioning a set of consumable resources amongcustomer-specific reservation sets 1104(1), 1104(2), 1104(3), 1104(4),and 1104(5) (collectively, customer-specific reservation sets 1104). Asshown, the set of consumable resources is apportioned to thecustomer-specific reservation sets 1104 according to a distribution1102.

Each of the customer-specific reservation sets 1104 is an aggregation ofactive reservations for a particular customer (for each of fivecustomers as shown). In particular, the customer-specific reservationset 1104(1) includes reservations 1114 a, 1114 b, and 1114 c. Thecustomer-specific reservation set 1104(2) includes reservations 1114 dand 1114 e. The customer-specific reservation set 1104(3) includesreservations 1114 f, 1114 g, 1114 h, and 1114 i. The customer-specificreservation set 1104(4) includes reservation 1114 j. Finally, thecustomer-specific reservation set 1104(5) includes reservation 1114 k.For convenient reference, the reservations 1114 a-1114 k may bereferenced collectively as reservations 1114. The reservations 114 eachhave a priority level associated therewith. In a typical embodiment, thepriority level may be selected from a plurality of priority levels ofincreasing priority (e.g., one to nine).

In various embodiments, the distribution 1102 may be applied asdescribed, for example, with respect to the number-to-percentagedistribution 908 of FIG. 9 or the stationary distribution 704 of FIG. 7.In that way, the distribution 1102 may apply a balanced partition schemeas described with respect to FIG. 7 or apply a high-priority superpartition and a regular-priority super partition as described withrespect to FIG. 9. Thus, the distribution 1102 distributescustomer-specific resource percentages 1103(1)-1103(5) to thecustomer-specific reservation sets 1104(1)-1104(5), respectively. Forconvenient reference, the customer-specific resource percentages1103(1)-1103(5) may be referenced collectively as resource percentages1103.

In a typical embodiment, after the distribution 1102 has been applied,stationary distributions 1106(1)-1106(5) are applied (collectively,stationary distributions 1106). The stationary distributions 1106apportion the customer-specific resource percentages 1103 to prioritygroupings 1108 a-1108 g of the customer-specific reservation sets 1104.In a typical embodiment, a priority grouping exists for each prioritylevel at which there is an active reservation for a given customer. Inparticular, the stationary distributions 1106(1), 1106(2), 1106(3),1106(4), and 1106(5) serve to distribute priority-specific resourcepercentages 1110 a-1110 b, 1110 c, 1110 d-1110 e, 1110 f, and 1110 grespectively. In a typical embodiment, priority groupings of thepriority groupings 1108 that represent higher priority levels areawarded greater proportions of the customer-specific resourcepercentages 1103 than those priority groupings representing lowerpriority levels. An example of how the priority-specific resourcepercentages can be computed will be described with respect to FIG. 12.

The stationary distribution 1106(1) apportions the customer-specificresource percentage 1103(1) to the priority groupings 1108 a and 1108 b.The priority groupings 1108 a and 1108 b indicate that thecustomer-specific reservation set 1104(1) includes one or morereservations at priority-level one and priority-level two, respectively(i.e. two priority groupings). The priority-specific resourcepercentages 1110 a and 1110 b thereby represent proportions of thecustomer-specific resource percentage 1103(1) that are apportioned tothe priority groupings 1108 a and 1108 b, respectively. According to thestationary distribution 1106(1), the priority-specific resourcepercentages 1110 a and 1110 b collectively sum to one.

The stationary distribution 1106(2) apportions the customer-specificresource percentage 1103(2) to the priority grouping 1108 c. Thepriority grouping 1108 c indicates that the customer-specificreservation set 1104(2) includes one or more reservations atpriority-level three (i.e., one priority grouping). Thepriority-specific resource percentage 1110 c thereby represents aproportion of the customer-specific resource percentage 1103(2) that isapportioned to the priority grouping 1108 c. Since the customer-specificreservation set 1104(2) only includes one priority grouping, i.e., thepriority grouping 1108 c, according to the stationary distribution1106(2), the priority-specific resource percentage 1110 c should equalone.

The stationary distribution 1106(3) apportions the customer-specificresource percentage 1103(3) to the priority groupings 1108 d and 1108 e.The priority groupings 1108 d and 1108 e indicate that thecustomer-specific reservation set 1104(3) includes one or morereservations at priority-level one and priority-level two, respectively(i.e. two priority groupings). The priority-specific resourcepercentages 1110 d and 1110 e thereby represent proportions of thecustomer-specific resource percentage 1103(3) that are apportioned tothe priority groupings 1108 d and 1108 e, respectively. According to thestationary distribution 1106(3), the priority-specific resourcepercentages 1110 d and 1110 e collectively sum to one.

The stationary distribution 1106(4) apportions the customer-specificresource percentage 1103(4) to the priority grouping 1108 f. Thepriority grouping 1108 f indicates that the customer-specificreservation set 1104(4) includes one or more reservations atpriority-level one (i.e., one priority grouping). The priority-specificresource percentage 1110 f thereby represents a proportion of thecustomer-specific resource percentage 1103(4) that is apportioned to thepriority grouping 1108 f. Since the customer-specific reservation set1104(4) only includes one priority grouping, i.e., the priority grouping1108 f, according to the stationary distribution 1106(4), thepriority-specific resource percentage 1110 f should equal one.

The stationary distribution 1106(5) apportions the customer-specificresource percentage 1103(5) to the priority grouping 1108 g. Thepriority grouping 1108 g indicates that the customer-specificreservation set 1104(5) includes one or more reservations atpriority-level one (i.e., one priority grouping). The priority-specificresource percentage 1110 g thereby represents a proportion of thecustomer-specific resource percentage 1103(5) that is apportioned to thepriority grouping 1108 g. Since the customer-specific reservation set1104(5) only includes one priority grouping, i.e., the priority grouping1108 g, according to the stationary distribution 1106(5), thepriority-specific resource percentage 1110 g should equal one.

After the stationary distributions 1106 are applied, stationarydistributions 1112(1), 1112(2), 1112(3), 1112(4), 1112(5), 1112(6), and1112(7) are applied (collectively, stationary distributions 1112). Thestationary distributions 1112 apportion the priority-specific resourcepercentages 1110 to individual reservations of the priority groupings1108. Specifically, the stationary distributions 1112(1), 1112(2),1112(3), 1112(4), 1112(5), 1112(6), and 1112(7) serve to distributereservation-specific resource percentages 1116 a, 1116 b-1116 c, 1116d-1116 e, 1116 f-1116 g, 1116 h-1116 i, 1116 j, and 1116 k respectively.In a typical embodiment, the stationary distributions 1112 effect anequal distribution of the priority-specific resource percentages 1110across each priority grouping of the priority groupings 1108.

More particularly, the stationary distribution 1112(1) apportions thepriority-specific resource percentage 1110 a to the reservation 1114 a(i.e., the priority grouping 1108 a). In this fashion, thereservation-specific resource percentage 1116 a is distributed to thereservation 1114 a. The reservation-specific resource percentage 1116 athereby represents a proportion of the priority-specific resourcepercentage 1110 a. Since the priority grouping 1108 a only includes onereservation (i.e., the reservation 1114 a), according to the stationarydistribution 1112(1), the reservation-specific resource percentage 1116a should equal one.

The stationary distribution 1112(2) apportions the priority-specificresource percentage 1110 b to the reservations 1114 b-1114 c (i.e., thepriority grouping 1108 b). In this fashion, the reservation-specificresource percentages 1116 b and 1116 c are distributed to thereservations 1114 b and 1114 c, respectively. The reservation-specificresource percentages 1116 b and 1116 c thereby represent proportions ofthe priority-specific resource percentage 1110 b that, according to thestationary distribution 1112(2), collectively sum to one.

The stationary distribution 1112(3) apportions the priority-specificresource percentage 1110 c to the reservations 1114 d-1114 e (i.e., thepriority grouping 1108 c). In this fashion, the reservation-specificresource percentages 1116 d and 1116 d are distributed to thereservations 1114 d and 1114 e, respectively. The reservation-specificresource percentages 1116 d and 1116 e thereby represent proportions ofthe priority-specific resource percentage 1110 c that, according to thestationary distribution 1112(3), collectively sum to one.

The stationary distribution 1112(4) apportions the priority-specificresource percentage 1110 d to the reservations 1114 f-1114 g (i.e., thepriority grouping 1108 d). In this fashion, the reservation-specificresource percentages 1116 f and 1116 g are distributed to thereservations 1114 f and 1114 g, respectively. The reservation-specificresource percentages 1116 f and 1116 g thereby represent proportions ofthe priority-specific resource percentage 1110 d that, according to thestationary distribution 1112(4), collectively sum to one.

The stationary distribution 1112(5) apportions the priority-specificresource percentage 1110 e to the reservations 1114 h-1114 i (i.e., thepriority grouping 1108 e). In this fashion, the reservation-specificresource percentages 1116 h and 1116 i are distributed to thereservations 1114 h and 1114 i, respectively. The reservation-specificresource percentages 1116 h and 1116 i thereby represent proportions ofthe priority-specific resource percentage 1110 e that, according to thestationary distribution 1112(5), collectively sum to one.

The stationary distribution 1112(6) apportions the priority-specificresource percentage 1110 f to the reservation 1114 j (i.e., the prioritygrouping 1108 f). In this fashion, the reservation-specific resourcepercentage 1116 j is distributed to the reservation 1114 j. Thereservation-specific resource percentage 1116 j thereby represents aproportion of the priority-specific resource percentage 1110 f. Sincethe priority grouping 1108 f only includes one reservation (i.e., thereservation 1114 j), according to the stationary distribution 1112(6),the reservation-specific resource percentage 1116 j should equal one.

The stationary distribution 1112(7) apportions the priority-specificresource percentage 1110 g to the reservation 1114 k (i.e., the prioritygrouping 1108 g). In this fashion, the reservation-specific resourcepercentage 1116 k is distributed to the reservation 1114 k. Thereservation-specific resource percentage 1116 k thereby represents aproportion of the priority-specific resource percentage 1110 g. Sincethe priority grouping 1108 g only includes one reservation (i.e., thereservation 1114 k), according to the stationary distribution 1112(7),the reservation-specific resource percentage 1116 k should equal one.

In a typical embodiment, the stationary distributions 1112 effect anequal distribution of the priority-specific resource percentages 1110across each priority grouping of the priority groupings 1108. Forexample, the priority groupings 1108 a, 1110 f, and 1110 g each includeone reservation. Therefore, the reservation-specific resourcepercentages 1116 a, 1116 j, and 1116 j should each equal one. By way offurther example, the priority groupings 1108 b, 1108 c, 1108 d, and 1108e each include two reservations. Therefore, the reservation-specificresource percentages 1116 b-1116 i should each equal one-half.

After the stationary distributions 1112 are applied, effectivedistributions 1118 a-1118 k are computed (collectively, effectivedistributions 1118). As explained above, the reservation-specificresource percentages 1116 are percentages of the priority-specificresource percentages 1110 that should be allocated to the reservations1114. The effective distributions 1118 are, in effect, a translation ofthe reservation-specific resource percentages 1116 into percentages ofthe current set of consumable resources. The effective distributions1118 a-1118 k are computed relative to the reservations 1114 a-1114 k,respectively.

The distribution 1102 may be considered an outer stationary distributionsince it is across customers and thus “outside” any one customer. Thestationary distributions 1106 and the stationary distributions 1108 maybe considered inner stationary distributions as since they are computedwith respect to particular customers and are thus “inside” particularcustomers. Each of the effective distributions 1118 represents aproportion of the current set of consumable resources that is beingapportioned to a given reservation of the reservations 1114. Each of theeffective distributions 1118 can be computed as a product of the outerstationary distribution and each inner stationary distribution.

Stated differently, each of the effective distributions 1118 can becomputed as a product of a corresponding reservation-specific resourcepercentage (from the reservation-specific resource percentages 1116), acorresponding priority-specific resource percentage (from thepriority-specific resource percentages 1110), and a correspondingcustomer-specific resource percentage (from the customer-specificresource percentages 1103). For example, the effective distribution 1118a can be computed as a product of the reservation-specific resourcepercentage 1116 a, the priority-specific resource percentage 1110 a, andthe customer-specific resource percentage 1103(1). It should beappreciated that the effective distributions 1118 should sum to one.

Once the effective distributions 1118 have been calculated, in a typicalembodiment, flow-control clocking weights 1120 a-1120 k are calculated(collectively, flow-control clocking weights 1120). The flow-controlclocking weights 1120 are, in effect, a translation of the effectivedistributions 1118 into defined quantities of resources that should beallocated to each of the reservations 1114. As explained in detailbelow, the flow-control clocking weights 1120 can be calculated asproducts of the effective distributions 1118 and a total number ofresources in the current set of consumable resources (i.e., N).

In particular, the flow-control clocking weight 1120 a equals theeffective distribution 1118 a multiplied by N. The flow-control clockingweight 1120 b equals the effective distribution 1118 b multiplied by N.The flow-control clocking weight 1120 c equals the effectivedistribution 1118 c multiplied by N. The flow-control clocking weight1120 d equals the effective distribution 1118 d multiplied by N. Theflow-control clocking weight 1120 e equals the effective distribution1118 e multiplied by N. The flow-control clocking weight 1120 f equalsthe effective distribution 1118 f multiplied by N. The flow-controlclocking weight 1120 g equals the effective distribution 1118 gmultiplied by N. The flow-control clocking weight 1120 h equals theeffective distribution 1118 h multiplied by N. The flow-control clockingweight 1120 i equals the effective distribution 1118 i multiplied by N.The flow-control clocking weight 1120 j equals the effectivedistribution 1118 j multiplied by N. The flow-control clocking weight1120 k equals the effective distribution 1118 k multiplied by N.

Each of the flow-control clocking weights 1120 is representative of adefined number of resources from the current set of consumableresources. As illustrated, a sum 1122 of the flow-control clockingweights 1120 equals the total number of consumable resources (i.e., N).Therefore, each of the flow-control clocking weights may be expressed infractional units of resources. As described in greater detail withrespect to FIGS. 13-14, the flow-clocking weights 1120 enable thebalanced-utilization partitioning scheme to be precisely executed on aper reservation basis.

After reviewing the inventive principles contained herein, one ofordinary skill in the art will appreciate that outer stationarydistributions and inner stationary distributions may be establisheddifferently than is described with respect to FIG. 11. For example, anynumber of inner stationary distributions may be established to furthersub-divide a given percentage of consumable resources. In each case,effective distributions may be computed as products of an outerstationary distribution and each inner stationary distribution.

FIG. 12 illustrates a process 1200 for calculating an inner stationarydistribution for a customer based on priorities. In a typicalembodiment, the process 1200 is performed for each active customer asdescribed, for example, with respect to FIG. 11. The process 1200 can beperformed by a computer cluster such as, for example, the computercluster 100 of FIG. 1. The process 1200 begins at step 1202. At step1202, the customer's priority groupings are identified. In a typicalembodiment, a priority grouping is identified for each priority level ofa plurality of priority levels at which the customer has an activereservation. The plurality of priority levels may be enumerated from 1to n. From step 1202, the process 1200 proceeds to step 1204.

At step 1204, a relative size between priorities is computed for eachpriority grouping. The relative size represents a proportion of a givenset of resources that should be apportioned a given priority level. Forexample, in one embodiment, the relative size can be calculated using anexponential function such as, for example, the function below, where prepresents a priority level of the priority grouping and F(p) representsthe relative size:F(p)=1.25^(p-1)From step 1204, the process 1200 proceeds to step 1206.

At step 1206, a sum of all relative sizes is calculated. From step 1206,the process 1200 proceeds to step 1208. At step 1208, for each prioritygrouping, a normalized relative size is calculated based on whichpriority groupings exist. For example, for each priority grouping, thenormalized relative size can equal the relative size divided by the sumof all relative sizes. It should appreciated that a sum of allnormalized relative sizes for the customer should equal one (i.e., astationary distribution). After step 1208, the process 1200 ends.

FIG. 13 illustrates clocking functionality 1300 of a flow-controlinstance 1328. The clocking functionality 1300 may be performed, forexample, as part of step 406 of the process 400 of FIG. 4. In a typicalembodiment, the flow-control instance 1328 operates as described withrespect to the flow-control instance 300 of FIG. 3. More particularly,however, the flow-control instance 1328 maintains reservations 1316(1),1316(2), and 1316(3). The reservations 1316(1), 1316(2), and 1316(3)include tasks 1320(1), 1320(2), and 1320(3), respectively, flow-controlclocking weights 1322(1), 1322(2), and 1322(3), respectively, andwait-times 1324(1), 1324(2), and 1324(3), respectively.

For convenient reference, the reservations 1316(1), 1316(2), and1316(3), the flow-control clocking weights 1322(1), 1322(2), and1322(3), and the wait-times 1324(1), 1324(2), and 1324(3) may bereferenced collectively as a current set of active reservations 1316,clocking weights 1322, and wait-times 1324, respectively. Forillustrative purposes, the current set of active reservations 1316 isshown to include the reservations 1316(1), 1316(2), and 1316(3), whichreservations may be considered to represent an exemplarysnapshot-in-time of the current set of active reservations 1316. Asdescribed in more detail below, the flow-control instance 1328 maintainsa priority queue 1318 of the current set of active reservations 1316.

The tasks 1320(1), 1320(2), and 1320(3) are each a derived grouping ofone or more tasks having, for example, a same task key as describedabove. In a typical embodiment, the clocking weights 1322 enable preciseexecution of a balanced-utilization partitioning scheme such as, forexample, the balanced-utilization partitioning schemes described withrespect to FIGS. 6-12. The flow-control clocking weights 1322(1),1322(2), and 1322(3) define a proportion of a current set of consumableresources that should be allocated to the reservations 1316(1), 1316(2),and 1316(3), respectively. As described with respect to FIG. 14, each ofthe flow-control clocking weights 1316 is computed as a product of aneffective distribution and a number of resources in the current set ofconsumable resources. Therefore, the clocking weights 1322(1), 1322(2),and 1322(3) may be expressed in fractional units of resources. It shouldbe appreciated that the clocking weights 1322(1), 1322(2), and 1322(3)change dynamically responsive to changes in, for example, the currentset of active reservations 1316.

The wait-times 1324 are an up-to-date measure of how needy thereservations 1316 are for the class of consumable resources and areupdated by the flow-control instance 1328. The wait-times 1324 may beexpressed in fractional units of resource-time (e.g., resource-secondswhen wall-time is used). The flow-control instance 1328 maintains avirtual clock 1326 that, every clocking cycle, initiates a virtualpendulum for each reservation in the current set of active reservations1316. In various embodiments, a clocking cycle can be, for example, aconfigurable unit of wall time (e.g., seconds or milliseconds) or aconfigurable number of CPU cycles. Other alternatives that can beconsidered a clocking cycle will be apparent to one of ordinary skill inthe art after reviewing the inventive principles described herein.

As described in greater detail with respect to FIGS. 15-16, thewait-times 1324(1), 1324(2), and 1324(3) are each given initial valuesupon creation of the reservations 1316(1), 1316(2), and 1316(3),respectively. The initiation of the virtual pendulums results in thereservations 1316 being “clocked” each clocking cycle according to theclocking weights 1322. It should be noted that each reservation in thecurrent set of active reservations 1316, by virtue of its inclusion,includes at least one task that has not been processed to completion.Therefore, in various embodiments in which lower wait-times representgreater neediness, the wait-times 1324 are decremented each clockingcycle by a factor of a corresponding clocking weight of the clockingweights 1322 (i.e., clocked to move the virtual pendulum in a negativedirection). An example of how the wait-times 1324 can be decrementedwill be described with respect to FIG. 14.

When a task of a given reservation in the current set of reservations1316 is finished accessing a given consumable resource, a correspondingwait-time of the wait-times 1324 is incremented by a factor of acorresponding clocking weight of the clocking weights 1322 (i.e.,clocked to move the virtual pendulum in a positive direction). Forexample, the corresponding wait-time can be incremented by a product ofthe corresponding clocking weight and a number of clocking units thatthe task has accessed the given consumable resource. In particular, inembodiments in which wall time is used to define the clocking cycle, thenumber of clocking units may be expressed in seconds with precision, forexample, to a nearest millisecond. Therefore, if the task has accessedthe given consumable resource for 12.001 seconds, the correspondingwait-time would be incremented by a product of 12.001 seconds and thecorresponding clocking weight (i.e., units of resource-seconds).

The flow-control instance 1328 maintains the priority queue 1318 basedon the current set of active reservations 1316. In a typical embodiment,the priority queue 1318 is sorted by the wait-times 1324, where a lowestwait-time indicates a most-needy reservation and a highest wait-timeindicates a least-needy reservation. Each clocking cycle, as the virtualpendulums are clocked negatively or positively in the manner describedabove, the flow-control instance 1328 re-sorts the priority queue 1318based on updated values for the wait-times 1324. As described in greaterdetail below with respect to the ensuing Figures, the flow-controlinstance 1328 assigns consumable resources of the current set ofconsumable resources to reservations of the current set of activereservations 1316 based on relative neediness. In a typical embodiment,the clocking weights 1322 and the wait-times 1324 may be maintained in adouble-precision floating-point format according to the IEEE 754standard.

FIG. 14 illustrates a process 1400 for decrementing a wait-time for eachreservation of a set of active reservations. For example, the process1400 may be performed as part of performing clocking as described withrespect to FIG. 13. The process 1400 may be performed by a computercluster such as, for example, the computer cluster 100 of FIG. 1. Theprocess 1400 is typically performed each clocking cycle. The process1400 begins at step 1402.

At step 1402, a flow-control clocking weight is accessed. In a typicalembodiment, the flow-control clocking weight is computed as part ofgenerating a balanced-utilization partitioning scheme as described withrespect to FIGS. 7, 9, and 11. As described above, the flow-controlclocking weight equals the reservation's effective distributionmultiplied by a total number of resources in a current set of consumableresources. In that way, the flow-control clocking weight represents anumber of resources and can be expressed in fractional units ofresources. From step 1402, the process 1400 proceeds to step 1404. Atstep 1404, for each resource, resource utilization during the clockingcycle is determined. For example, the computer cluster tracks how manyclocking units that each resource has been utilized. The resourceutilization can be expressed in fractional units of resource-time. Forexample, in embodiments in which wall time is utilized, the resourceutilization can be expressed in fractional units of resource-seconds toa nearest millisecond. From step 1404, the process 1400 proceeds to step1406.

At step 1406, resource utilizations are summed across the current set ofconsumable resources. From step 1406, the process 1500 proceeds to step1408. At step 1408, a maximum theoretical utilization of the current setof consumable resources is determined. The maximum theoreticalutilization corresponds to a number of clocking units that,collectively, the current set of consumable resources could have beenutilized during the clocking cycle. In a typical embodiment, the maximumtheoretical utilization equals the total number of resources multipliedby a length of the clocking cycle (e.g., seconds if wall time isutilized). Therefore, the maximum theoretical utilization also has unitsof resource-time (e.g., resource-seconds if wall time is utilized) Fromstep 1408, the process 1400 proceeds to step 1410.

At step 1410, an actual average capacity of the current set ofconsumable resources is computed. The actual average capacity can equal,for example, the summed resource utilizations divided by the maximumtheoretical utilization. Consequently, the actual average capacity canbe represented as a percentage of the maximum theoretical utilization.From step 1410, the process 1400 proceeds to step 1412. At step 1412,the computer cluster computes, for each reservation, a product of theflow-control clocking weight (units of resources), the actual averagecapacity, and the length of the clocking cycle (time, e.g., in seconds).Therefore, the product can be expressed in resource-seconds From step1412, the process 1400 proceeds to step 1414. At step 1414, for eachreservation, the wait-time is decremented by the product computed atstep 1412. After step 1414, the process 1400 ends.

FIG. 15 illustrates a collection 1500 of interactive processes that maybe executed by a flow-control instance. The flow-control instancemanages a dynamically changing set of consumable resources relative to amanaged task type requiring utilization of a class of consumableresources. In a typical embodiment, the collection 1500 is used tocontrol sharing of the set of consumable resources among a dynamicallychanging set of active customers. The collection 1500 may be executed bya computer cluster such as, for example, the computer cluster 100 ofFIG. 1.

The collection 1500 includes a customer-needs process 1502, areservations process 1504, a reservation-accounting process 1506, and aresource-allocation process 1508. In a typical embodiment, each processin the collection 1500 is executed in parallel by one or more computersof the computer cluster. Upon instantiation of the flow-controlinstance, no process of the collection 1500 is typically in execution.Execution of the collection 1500 typically begins with a first executionof the customer-needs process 1502. The customer-needs process 1500begins at step 1502(1).

At step 1502(1), a customer-specific task group is created. In a typicalembodiment, the customer-specific task group is created responsive to arequest for service being received from a customer. Creation of thecustomer-specific task group typically involves grouping tasks, forexample, by task key as described above with respect to FIGS. 3-5. Itshould be appreciated that the customer-specific task group mightinclude a single task or, for example, many thousands of tasks. Fromstep 1502(1), execution proceeds to step 1502(2) of the customer-needsprocess 1502.

At step 1502(2), it is determined whether the customer is a firstcustomer in the set of active customers. If so, concurrently: (1) thereservation-accounting process 1506 is initiated; (2) theresource-allocation process 1508 is initiated; and (3) thecustomer-needs process proceeds to step 1502(3). If it is determined atstep 1502(2) that the customer is not the first customer in the set ofactive customers, execution proceeds to step 1502(3). At step 1502(3),it is determined whether a total number of active customers has changed.If not, the customer-needs process 1502 proceeds to step 1504(1) of thereservations process 1504. If it is determined at step 1502(3) that thetotal number of active customers has changed, execution proceeds to step1502(4).

At step 1502(4), an outer stationary distribution is computed for allactive customers. The outer stationary distribution may be computed, forexample, as described with respect to FIGS. 7 and 9. Accordingly, foreach active customer, computing the outer stationary distributioninvolves computing, for each active customer, a customer-resourcepercentage according to a proportion of the current set of consumableresources that should be allocated to the active customer. All suchcustomer-resource percentages should generally sum to one. In a typicalembodiment, step 1502(4) results in a balanced-utilization partitioningscheme as described with respect to FIGS. 6-10. More particularly, thebalanced-utilization partitioning scheme includes one partition for eachactive customer. From step 1502(4), execution proceeds to step 1502(5).

At step 1502(5), an effective distribution is computed for eachreservation. The effective distribution can be computed as describedwith respect to FIGS. 7 and 9. From step 1502(5), execution proceeds tostep 1502(6). At step 1502(6), a flow-control clocking weight iscalculated for each reservation. In a typical embodiment, the effectivedistribution is a primary component of each reservation's flow-controlclocking weight. As described with respect to FIGS. 7, 9, 11, and 13,each reservation's flow-control clocking weight can be calculated as aproduct of the effective distribution and a total number of consumableresources in the current set of consumable resources. From step 1502(6),execution proceeds to step 1504(1) of the reservations process 1504.

The reservations process 1504 begins at step 1504(1). At step 1504(1), anew reservation is added to the priority queue. The new reservationcorresponds to the customer-specific task group created at step 1502(1)of the customer-needs process 1502. From step 1504(1), executionproceeds to step 1504(2). At step 1504(2), an initial value is assignedto a wait-time for the new reservation. As described with respect toFIG. 13, the flow-control instance maintains wait-times for eachreservation in the priority queue. The wait-times are considered ameasure of how needy the reservations are for the class of consumableresources. The initial value of the wait-time for the new reservationcan be, for example, a negative value of the new reservation's effectivedistribution (computed at step 1502(5)). In various other embodiments,the initial value of the wait-time can be, for example, a negative valueof the new reservation's flow-control clocking weight (computed at step1502(6)).

The reservation-accounting process 1506 begins at step 1506(1). At step1506(2), the flow-control instance waits one clocking cycle (e.g., onesecond). From step 1506(2), execution proceeds to step 1506(3). At step1506(3), the wait-time for each reservation in the priority queue isupdated based on the reservation's flow-control clocking weight. Theupdate yields a new value for the wait-time. In a typical embodiment,for each reservation, this update effects a clocking of the reservationin a negative direction as described with respect to FIG. 13. An exampleof how the wait-time for each reservation can be clocked negatively isdescribed with respect to FIG. 14. From step 1506(3), execution proceedsto step 1506(4). At step 1506(4), the priority queue is reordered bywait-time, where a lowest wait-time indicates a most-needy reservationand a highest wait-time indicates a least-needy reservation. From step1506(4), execution returns to step 1506(2) and proceeds as describedabove.

The resource-allocation process 1508 begins at step 1508(1). At step1508(2), it is determined whether any consumable resources in thecurrent set of consumable resources are free (i.e., available for use).If not, the resource-allocation process 1508 remains at step 1508(2)until a consumable resource in the current set of consumable resourcesis free. If it is determined at step 1508(2) that a consumable resourcein the current set of consumable resources is free, execution proceedsto step 1508(3). At step 1508(3), the consumable resource is marked as“in use.” From step 1508(3), execution proceeds to step 1506(5) of thereservation-accounting process 1506.

At step 1506(5), the consumable resource is assigned to a selectedreservation in the priority queue that is qualified to use theconsumable resource. In a typical embodiment, the selected reservationis the most-needy reservation as defined by having the lowest wait-timein the priority queue. At this point, the selected reservation isgranted access to the assigned consumable resource until the selectedreservation is finished, as described in greater detail below. From step1506(5), execution proceeds to step 1504(6) of the reservations process1504.

At step 1504(6), it is determined whether the selected reservation isstill active. In a typical embodiment, the selected reservation is stillactive if the selected reservation includes at least one task that hasnot been processed to completion. If not, execution proceeds to step1504(7). At step 1504(7), the selected reservation is removed from thepriority queue. From step 1504(7), execution proceeds to step 1502(3)and continues as described above. If it is determined at step 1504(6)that the selected reservation is still active, execution proceeds tostep 1504(3).

At step 1504(3), the selected reservation's wait-time is temporarilyupdated based on an average utilization time. The average utilizationtime can be represented in units of resource-time (i.e., as a product ofresources and time). If wall time is utilized, the units ofresource-time could, more specifically, be considered resource-seconds.The average utilization time is an average number of resource-time units(e.g., resource-seconds) that tasks grouped under the selectedreservation's task key have utilized a consumable resource beforeyielding back the consumable resource. The average utilization time isbased on previous consumable-resource assignments. If the selectedreservation has not previously been assigned a consumable resource, theaverage utilization time may be a default value such as, for example,one resource-second. In a typical embodiment, a product of the averageutilization time and the flow-control clocking weight is added to theselected reservation's wait-time. In a typical embodiment, this updateprevents the selected reservation from remaining at the top of thepriority queue. From step 1504(3), execution proceeds to step 1504(4).

At step 1504(4), it is determined whether a task of the selectedreservation is finished with the assigned consumable resource. Executionremains at step 1504(4) until a currently executing task is finished. Aseach task of the selected reservation is finished, execution proceeds tostep 1504(5). At step 1504(5), the selected reservation's wait-time isadjusted based on an actual utilization time (i.e., resource-time) ofthe finished task. The adjustment typically involves subtracting theproduct temporarily applied at step 1504(3) and, in its place, adding aproduct of the actual utilization time and one resource, therebyyielding units of resource-time (e.g., resource-seconds). For example,in various embodiments in which the actual utilization time is expressedin fractional units of wall time, the actual utilization time is preciseto a nearest millisecond. In a typical embodiment, the averageutilization time for the selected reservation is also updated based onthe actual utilization time. From step 1504(5), execution proceeds tostep 1504(6) and proceeds as described above.

FIG. 16 illustrates a collection 1600 of interactive processes that maybe executed by a flow-control instance such as, for example, theflow-control instance 200 of FIG. 2A. The flow-control instance managesa dynamically changing set of consumable resources relative to a managedtask type requiring utilization of a class of consumable resources. In atypical embodiment, the collection 1600 is used to control sharing ofthe set of consumable resources among a dynamically changing set ofactive customers. The collection 1600 may be executed by a computercluster such as, for example, the computer cluster 100 of FIG. 1.

In similar fashion to the collection 1500 of FIG. 15, the collection1600 includes a customer-needs process 1602, a reservations process1604, a reservation-accounting process 1606, and a resource-allocationprocess 1608. Differently than the collection 1500, the collection 1600additionally includes a customer-priorities process 1603. In a typicalembodiment, each process in the collection 1600 is executed in parallelby one or more computers of the computer cluster. Upon instantiation ofthe flow-control instance, no process of the collection 1600 istypically in execution. Execution of the collection 1600 typicallybegins with a first execution of the customer-needs process 1602. Thecustomer-needs process 1600 begins at step 1602(1).

At step 1602(1), a customer-specific task group is created. In a typicalembodiment, the customer-specific task group is created responsive to arequest for service being received from a customer. Creation of thecustomer-specific task group typically involves grouping tasks, forexample, by task key as described above with respect to FIGS. 3-5. Itshould be appreciated that the customer-specific task group mightinclude a single task or, for example, many thousands of tasks. Fromstep 1602(1), execution proceeds to step 1602(2).

At step 1602(2), it is determined whether the customer is a firstcustomer in the set of active customers. If so, concurrently: (1) thereservation-accounting process 1606 is initiated; (2) theresource-allocation process 1608 is initiated; and (3) thecustomer-needs process proceeds to step 1602(3). If it is determined atstep 1602(2) that the customer is not the first customer in the set ofactive customers, execution proceeds to step 1602(3). At step 1602(3),it is determined whether a total number of active customers has changed.If not, the customer-needs process 1602 proceeds to step 1604(1) of thereservations process 1604. If it is determined at step 1602(3) that thetotal number of active customers has changed, execution proceeds to step1602(4).

At step 1602(4), an outer stationary distribution is computed for allactive customers. The outer stationary distribution may be computed, forexample, as described with respect to FIGS. 7, 9, and 11. Accordingly,for each active customer, computing the outer stationary distributioninvolves assigning, to each active customer, a customer-specificresource percentage according to a proportion of the current set ofconsumable resources that should be allocated to the active customer.All such customer-specific resource percentages should generally sum toone. From step 1602(4), execution proceeds to step 1603(1) of thecustomer-priorities process 1603.

The customer-priorities process 1603 begins at step 1603(1). At step1603(1), an inner stationary distribution is computed for each activecustomer. Computation of the inner stationary distribution involvescalculating a priority-based resource percentage for each prioritygrouping of the customer's active reservations. A priority groupingexists for each priority level at which the customer has activereservations. All such priority-based resource percentages should sum toone. The inner stationary distribution may be computed, for example, asdescribed with respect to FIGS. 11-12.

In a typical embodiment, step 1603(1) results in a balanced-utilizationpartitioning scheme as described with respect to FIGS. 6-12. Moreparticularly, the balanced-utilization partitioning scheme includes onepartition for each priority grouping so that a total number ofpartitions equals a number of priority groupings across all customers.From step 1603(1), execution proceeds to step 1603(2). At step 1603(2),an effective distribution is computed for each reservation. Theeffective distribution can be computed as described with respect toFIGS. 11-12. From step 1603(2), execution proceeds to step 1603(3).

At step 1603(3), a flow-control clocking weight is calculated for eachreservation. In a typical embodiment, the effective distribution is aprimary component of each reservation's flow-control clocking weight. Asdescribed with respect to FIGS. 7, 9, 11, and 13, each reservation'sflow-control clocking weight can be calculated as a product of theeffective distribution and a total number of consumable resources in thecurrent set of consumable resources. From step 1603(3), executionproceeds to step 1604(1) of the reservations process 1604.

The reservations process 1604 begins at step 1604(1). At step 1604(1), anew reservation is added to the priority queue. The new reservationcorresponds to the customer-specific task group created at step 1602(1)of the customer-needs process 1602. From step 1604(1), executionproceeds to step 1604(2). At step 1604(2), an initial value is assignedto a wait-time for the new reservation. As described with respect toFIG. 13, the flow-control instance maintains wait-times for eachreservation in the priority queue. The wait-times are considered ameasure of how needy the reservations are for the class of consumableresources. The initial value of the wait-time for the new reservationcan be, for example, a negative value of the new reservation's effectivedistribution (computed at step 1603(2)). In various other embodiments,the initial value of the wait-time can be, for example, a negative valueof the new reservation's flow-control clocking weight (computed at step1603(3)).

The reservation-accounting process 1606 begins at step 1606(1). At step1606(2), the flow-control instance waits one clocking cycle (e.g., onesecond). From step 1606(2), execution proceeds to step 1606(3). At step1606(3), the wait-time for each reservation in the priority queue isupdated based on the reservation's flow-control clocking weight. Thisupdate yields a new value for the wait-time. In a typical embodiment,for each reservation, this update also effects a clocking of thereservation in a negative direction as described with respect to FIG.13. An example of how the wait-time for each reservation can be clockednegatively is described with respect to FIG. 14. From step 1606(3),execution proceeds to step 1606(4). At step 1606(4), the priority queueis reordered by wait-time, where a lowest wait-time indicates amost-needy reservation and a highest wait-time indicates a least-needyreservation. From step 1606(4), execution returns to step 1606(2) andproceeds as described above.

The resource-allocation process 1608 begins at step 1608(1). At step1608(2), it is determined whether any consumable resources in thecurrent set of consumable resources are free (i.e., available for use).If not, the resource-allocation process 1608 remains at step 1608(2)until a consumable resource in the current set of consumable resourcesis free. If it is determined at step 1608(2) that a consumable resourcein the current set of consumable resources is free, execution proceedsto step 1608(3). At step 1608(3), the consumable resource is marked as“in use.” From step 1608(3), execution proceeds to step 1606(5) of thereservation-accounting process 1606.

At step 1606(5), the consumable resource is assigned to a selectedreservation in the priority queue that is qualified to use theconsumable resource. In a typical embodiment, the selected reservationis the most-needy reservation as defined by having the lowest wait-timein the priority queue. At this point, the selected reservation isgranted access to the assigned consumable resource until the selectedreservation is finished, as described in greater detail below. From step1606(5), execution proceeds to step 1604(6) of the reservations process1604.

At step 1604(6), it is determined whether the selected reservation isstill active. In a typical embodiment, the selected reservation is stillactive if the selected reservation includes at least one task that hasnot been processed to completion. If not, execution proceeds to step1604(7). At step 1604(7), the selected reservation is removed from thepriority queue. From step 1604(7), execution proceeds to step 1602(3)and continues as described above. If it is determined at step 1604(6)that the selected reservation is still active, execution proceeds tostep 1604(3).

At step 1604(3), the selected reservation's wait-time is temporarilyupdated based on an average utilization time. The average utilizationtime can be represented in units of resource-time (i.e., as a product ofresources and time). If wall time is utilized, the units ofresource-time could, more specifically, be considered resource-seconds.The average utilization time is an average number of resource-time units(e.g., resource-seconds) that tasks grouped under the selectedreservation's task key have utilized a consumable resource beforeyielding back the consumable resource. The average utilization time isbased on previous consumable-resource assignments. If the selectedreservation has not previously been assigned a consumable resource, theaverage utilization time may be a default value such as, for example,one resource-second. In a typical embodiment, a product of the averageutilization time and the flow-control clocking weight is added to theselected reservation's wait-time. In a typical embodiment, this updateprevents the selected reservation from remaining at the top of thepriority queue. From step 1604(3), execution proceeds to step 1604(4).

At step 1604(4), it is determined whether a task of the selectedreservation is finished with the assigned consumable resource. Executionremains at step 1604(4) until a currently executing task is finished. Aseach task of the selected reservation is finished, execution proceeds tostep 1604(5). At step 1604(5), the selected reservation's wait-time isadjusted based on an actual utilization time of the finished task. Theadjustment typically involves subtracting the product temporarilyapplied at step 1604(3) and, in its place, adding a product of theactual utilization time and ‘one’ resource, thereby yielding units ofresource-time (e.g., resource-seconds). For example, in variousembodiments in which the actual utilization time is expressed infractional units of wall time, the actual utilization time is precise toa nearest millisecond. In a typical embodiment, the average utilizationtime for the selected reservation is also updated based on the actualutilization time. From step 1604(5), execution proceeds to step 1604(6)and proceeds as described above.

For purposes of this disclosure, an information handling system mayinclude any instrumentality or aggregate of instrumentalities operableto compute, calculate, determine, classify, process, transmit, receive,retrieve, originate, switch, store, display, communicate, manifest,detect, record, reproduce, handle, or utilize any form of information,intelligence, or data for business, scientific, control, or otherpurposes. For example, an information handling system may be a personalcomputer (e.g., desktop or laptop), tablet computer, mobile device(e.g., personal digital assistant (PDA) or smart phone), server (e.g.,blade server or rack server), a network storage device, or any othersuitable device and may vary in size, shape, performance, functionality,and price. The information handling system may include random accessmemory (RAM), one or more processing resources such as a centralprocessing unit (CPU) or hardware or software control logic, ROM, and/orother types of nonvolatile memory. Additional components of theinformation handling system may include one or more disk drives, one ormore network ports for communicating with external devices as well asvarious input and output (I/O) devices, such as a keyboard, a mouse,touchscreen and/or a video display. The information handling system mayalso include one or more buses operable to transmit communicationsbetween the various hardware components.

Although various embodiments of the method and apparatus of the presentinvention have been illustrated in the accompanying Drawings anddescribed in the foregoing Detailed Description, it will be understoodthat the invention is not limited to the embodiments disclosed, but iscapable of numerous rearrangements, modifications and substitutionswithout departing from the spirit of the invention as set forth herein.

What is claimed is:
 1. A method of operating a computer cluster toapportion consumable resources by the computer cluster to process a setof active reservations, the method comprising: via at least one processon the computer cluster comprising a plurality of computers, identifyinga current set of the consumable resources that fulfill a resource needof a managed task type; accessing a stored listing of tasks on thecomputer cluster, wherein the tasks are a managed task type that utilizea class of the consumable resources; deriving, by the at least oneprocess, from the stored listing of tasks, a set of the activereservations of the managed task type, wherein (i) deriving the set ofactive reservations comprises grouping the tasks from the stored listingof tasks into a group based on at least a common characteristic of thetasks, (ii) each active reservation comprises the grouping of tasks fromthe stored listing of tasks which have a set of characteristics incommon, and (iii) the common characteristic of tasks in the group is acharacteristic of the tasks that exists prior to the grouping;apportioning, by the at least one process, the current set of consumableresources among the set of active reservations to yield abalanced-utilization partitioning scheme, the balanced-utilizationpartitioning scheme comprising a flow-control clocking weight for eachactive reservation, wherein: (i) the apportioning the current set ofconsumable resources among the set of active reservations comprisesassigning each active reservation to one or more of the consumableresources; (ii) the flow-control clocking weights comprise fractionalunits of the current set of consumable resources; and (iii) a sum of theflow-control clocking weights equals a total number of the consumableresources in the current set of consumable resources; and executingtasks associated with the current set of active reservations by at leastperforming controlled sharing of the current set of consumable resourcesamong the set of active reservations via the flow-control clockingweights.
 2. The method of claim 1, comprising performing controlledsharing of the current set of consumable resources among the set ofactive reservations via the flow-control clocking weights.
 3. The methodof claim 1, wherein the deriving comprises grouping tasks having a sametask key into a same reservation.
 4. The method of claim 3, wherein thetask key comprises a plurality of characteristics selected from thegroup consisting of: service ID, customer ID, and priority level.
 5. Themethod of claim 1, wherein each active reservation belongs to an activecustomer of a set of active customers.
 6. The method of claim 5, whereinthe apportioning comprises: computing a stationary distribution of thecurrent set of consumable resources across the set of active customers,the stationary distribution comprising distribution of an equalproportion of the current set of consumable resources to each activecustomer; for each active customer, calculating a reservation-specificresource percentage for each of the active customer's activereservations; and computing the flow-control clocking weights based onthe stationary distribution and the reservation-specific resourcepercentages.
 7. The method of claim 5, wherein the apportioningcomprises establishing a plurality of super partitions relative to thecurrent set of consumable resources.
 8. The method of claim 7, whereinthe establishing comprises: establishing a regular-priority superpartition comprising a first subset of the current set of consumableresources; and establishing a high-priority super partition comprising asecond subset of the current set of consumable resources.
 9. The methodof claim 8, wherein establishing a regular-priority super partitioncomprises reallocating unutilized resources of the high-priority superpartition to the regular-priority super partition.
 10. The method ofclaim 8, comprising: wherein establishing a regular-priority superpartition comprises establishing a low-priority super partition that isa subset of the regular-priority super partition; and wherein thelow-priority super partition is made available to a designated class ofcustomers when the regular-priority super partition is not fullyutilized.
 11. The method of claim 8, wherein the apportioning comprises:distributing an equal resource quantity of the regular-priority superpartition to each active customer; distributing a predetermined resourcequantity of the high-priority super partition to each active customerthat is eligible for the high-priority super partition; for each activecustomer, summing the equal resource quantity and any predeterminedresource quantity to yield a resource apportionment; for each activecustomer, calculating a reservation-specific resource percentage foreach of the active customer's active reservations; and computing theflow-control clocking weights based on the reservation-specific resourcepercentages and the resource apportionment.
 12. The method of claim 5,wherein the apportioning comprises: computing an outer distribution ofthe current set of consumable resources across the set of activecustomers; wherein the outer distribution comprises distribution of aresource percentage of the current set of consumable resources to eachactive customer; for each active customer, computing an inner stationarydistribution across the active customer's priority groupings, the innerstationary distribution comprising distribution of a priority-specificresource percentage to each priority grouping; for each active customer,calculating a reservation-specific resource percentage for each of theactive customer's active reservations; and computing the flow-controlclocking weights based on the outer distribution, the inner stationarydistribution, and the reservation-specific resource percentages.
 13. Themethod of claim 12, wherein the outer distribution comprises astationary distribution of the current set of consumable resourcesacross the set of active customers.
 14. The method of claim 12, wherein,for each active customer, computing an inner stationary distributioncomprises: identifying a priority grouping for each priority level of aplurality of priority levels at which the active customer has an activereservation; computing a relative size between priorities for eachpriority grouping; and for each priority grouping, calculating anormalized relative size, the normalized relative size comprising thepriority grouping's priority-specific resource percentage.
 15. Themethod of claim 1, wherein the method is repeated as new tasks of themanaged task type are created on the computer cluster.
 16. The method ofclaim 1, wherein the method is performed independently and in parallelby a plurality of flow-control instances resident on the computercluster.
 17. The method of claim 1, wherein the flow-control clockingweights comprise fractional units of resources and are maintained in adouble-precision floating-point format.
 18. The method of claim 1,wherein a sum of the flow-control clocking weights equals a number ofresources in the current set of consumable resources.
 19. An informationhandling system, comprising: a computer cluster comprising a pluralityof computers, the computer cluster having at least one flow-controlinstance resident and executing thereon; wherein, for each of the atleast one flow-control instance, to apportion consumable resources ofthe computer cluster to process a set of active reservations thecomputer cluster is programmed to: identify a current set of consumableresources that fulfill a resource need of a managed task type; access astored listing of tasks on the computer cluster; derive, from the storedlisting of tasks, a set of active reservations of the managed task type,wherein (i) to derive the set of active reservations comprises groupingthe tasks from the stored listing of tasks into a group based on atleast a common characteristic of the tasks, (ii) each active reservationcomprises the grouping of tasks from the stored listing of tasks whichhave a set of characteristics in common, and (iii) the commoncharacteristic of tasks in the group is a characteristic of the tasksthat exists prior to the grouping; apportion the current set ofconsumable resources among the set of active reservations to yield abalanced-utilization partitioning scheme, wherein the apportionment ofthe current set of consumable resources among the set of activereservations comprises assigning each active reservation to a singleconsumable resource, and the balanced-utilization partitioning schemecomprising a flow-control clocking weight for each active reservation,wherein: (i) the apportionment of the current set of consumableresources among the set of active reservations comprises assigning eachactive reservation to one or more of the consumable resources; (ii) theflow-control clocking weights comprise fractional units of the currentset of consumable resources; and (iii) a sum of the flow-controlclocking weights equals a total number of the consumable resources inthe current set of consumable resources; and execute tasks associatedwith the current set of active reservations by at least performingcontrolled sharing of the current set of consumable resources among theset of active reservations via the flow-control clocking weights. 20.The information handling system of claim 19, wherein the at least oneflow-control instance comprises a plurality of flow-control instancesthat execute independently and in parallel.
 21. A computer-programproduct comprising a non-transitory computer-usable medium havingcomputer-readable program code embodied therein, the computer-readableprogram code adapted to be executed to implement a method of operating acomputer cluster to apportion consumable resources of the computercluster to process a set of active reservations by performing operationscomprising: on a computer cluster comprising a plurality of computers,identifying a current set of consumable resources that fulfill aresource need of a managed task type; accessing a stored listing oftasks on the computer cluster; deriving, from the stored listing oftasks, a set of active reservations of the managed task type, wherein(i) deriving the set of active reservations comprises grouping the tasksfrom the stored listing of tasks into a group based on at least a commoncharacteristic of the tasks, (ii) each active reservation comprises thegrouping of tasks from the stored listing of tasks which have a set ofcharacteristics in common, and (iii) the common characteristic of tasksin the group is a characteristic of the tasks that exists prior to thegrouping; apportioning the current set of consumable resources among theset of active reservations to yield a balanced-utilization partitioningscheme, wherein the apportioning the current set of consumable resourcesamong the set of active reservations comprises assigning each activereservation to a single consumable resource, and thebalanced-utilization partitioning scheme comprising a flow-controlclocking weight for each active reservation, wherein: (i) theapportioning the current set of consumable resources among the set ofactive reservations comprises assigning each active reservation to oneor more of the consumable resources; (ii) the flow-control clockingweights comprise fractional units of the current set of consumableresources; and (iii) a sum of the flow-control clocking weights equals atotal number of the consumable resources in the current set ofconsumable resources; and executing tasks associated with the currentset of active reservations by at least performing controlled sharing ofthe current set of consumable resources among the set of activereservations via the flow-control clocking weights.